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1. Introduction 



In a Matrix-Differential Game, (MDG) a finite payoff matrix A is 
given, and player I (II) tries to maximize (minimize) the payoff by picking 
the proper row (column). However, the choices of the two players are not 
made simultaneously, so that the conventional result that the game value is 
obtained when the two players use optimal mixed strategies does not hold. 
Instead, a number a , with 0 ^ a ^ 1 , is given, with the interpretation 
that a is the relative reaction time of player I. The elements of A 
are interpreted as payoff r ate s, with the payoff being at the rate a(i,j) 
for a time a if player II has just picked column j (after which player 
I picks a new row) , or for a time 1-c if player I has just picked row i 
(after which player II picks a new column). Play is assumed to go on 
indefinitely, and the payoff ft(a) is defined to be the long run average 
of the payoff rate. The words u long run" are of course relative to the 
length of the time unit. 

The idea of an MDG was invented by Danskin [1], who observed that the 
ability to solve MDG's is a prerequisite for being able to solve for the 
ft-value of a differential game, where the payoff matrix of the MDG would 
be a Hamiltonian function involving the equations of motion. In fact, it 
is non-separable differential games that provide most of the interest in 
MDG’s, since it can be shown that optimal strategies for playing MDG f s 
do not involve choosing random numbers. The spectre of the two players 
trying to choose independent random numbers continuously in a differential 
game has given rise to the ^-formulation (along with several other [2], 

[3], and [A]), where the need for mixed strategies is eliminated by having 
the players "take turns" in some sense or other. 
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Formally, an N x M MDG is a stochastic game with perfect information, 
2MN positions (the position depends on whose move it is, hence the factor 
of 2), and non-zero stop probabilities. The existence of the ft-value 
and of stationary optimal strategies has therefore been proved by Gillete 
in his paper on the subject [5]. (His proof contained an error that was 
subsequently corrected by Liggett and Lippman [6]). Our Theorem 1 goes on 
to state that the Q-value and the optimal stationary strategies .must 
satisfy certain "equilibrium equations" (2.6 and 2.7) that form the basis 
of the computational procedure discussed in chapter 3. Theorem 2 shows 
that our ft-value is the same as Danskin : s, and Theorem 3 establishes some 
elementary properties for the ^2-value of a MDG. These three theorems, 
together with the computational procedure of chapter 3, are the content 
of this technical report. 
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2 . The Equilibrium Equations 



Throughout this report the symbol A = [a(i,j)] will denote an 

N x M matrix with real entries a (i, j ) ; i = 1, . . . ,N ; j = 1, . . . ,M. 

We further reserve the letters I and J to denote the sets of row and 

column indices, respectively, i.e. 

i = n,.. ,,n) , j = (i, . . . , m> . 

The letter O will always denote a real number from the interval [0,1] . 

Next let P be the set of all mappings from J to I , and let 
Q. be the set of all mappings from I to J . A sequence 

p = (p v : P v eP > ^=1,2, . . .} 

or a sequence 

Q = (q v •* q v e£, v=i,2,. . .} 

will be referred to as response strategy of player 1 or 2 respectively. 

oo oo 

The sets of all response strategies will be denoted P and Q. * A response 
strategy P or Q such that p^ = p^ = ... = p or such that q^ = q^ = . . . = q 
will be called stationary . 

r i oo r i r ^ 00 

Let P = ip^ieP and Q = iq^cQ. be a pair of response strategies, and 

let n be a positive integer. With any such pair (P , Q) we now associate 

a quantity » called the n-stage payoff given the row-predecesso r 

i cl , and defined by 
o y 



n (p > Q 1 1 ) = £ I Oa(i j v ) + (l-o)a(i v ,j v ) , 
v=l 



where i = q (i „ ) , i = p (i ) , V = 1,2,... 

J V V-l V V 

Similarly, the n-stage payoff given the column-predecessor j^cJ is defined by 



n ( p »Q|j 0 ) = £ l (i-crWWP + 0a( V V 



V=1 
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where this time 



\ = Pv (J v-l } ’ j v = W ’ v = • 

OO oo 

The triplet G(a) = (P 9 Q , A) with the sequences of payoffs defined 

as above will be referred to as matrix-dif f erential game . (MDG) 

We are now ready to state the basic definition. 

oo oo 

Def ini tion : If there exist a pair of response strategies P* e P , Q* € Q 



and a real number fi(a) such that for any i Q eI or j Q e ^ 

14m H n (P*,Q*|i 0 ) - lja H n (P*,Q*|j 0 ) - K(a) , 

and for any P cP , Qe() 

l^m sup H n (P,Q*|i Q ) < (2(a) , 

l^m sup H n (P,Q*|j o ) < (2(a) , 

P& in f H n (P*,Q|i o ) S (2(a) , 

l|m inf H n (P*,Q | 3 q ) ^ , 



( 2 . 1 ) 

( 2 . 2 ) 

(2.3) 

(2.4) 

(2.5) 



then £3(a) is called the omega-value and P* and Q* optimal response 
strategies of the MDG G(g). 



Lemma : Let 00 , x^, 

N + M equations : 



> V y r 



,y be a solution of the system of 
m 



x . + 00 = min[aa(i,j) + y.] , iel , (2.6a) 

1 jeJ 3 

y. + U)(l-a) = max[ (l-C)a (i, j )+x. ] , jeJ , (2.6b) 

J iel 1 

let p*eP and q*eQ. be such that 

x_ L + u0 = aa(i,q*(i)) + y , iel , (2.7a) 

Yj + w(l-c) = (l-a)a(p*(j),j) + x p *Q) » jeJ. (2.7b) 

Then (2.1) through (2.5) hold with (2(a) = ^ and stationary 



P* = {p*,p*,...} , Q* = {q*,q*,...} . 
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Proof : We begin with the inequality (2.2) . By (2.6b) we have for 



every k e I , j e J 

y. + co(l-a) > (l-a)a (k, j ) + x 



so that in particular for j = q*(i) 

Y q*(i) + a)(1 " Q) ~ (l-a)a(k,q*(i)) + x R . 

Substituting for yq*(-Q into (2.7a) this becomes 

x. + 030 > aa (i , q* (i) ) + (]-a)a(k,q*(i)) + x, - 03(1 -a) 

1 K. 



or 

( 1 ) > aa(i,q*(i)) + (l-a)a (k, q* (i) ) + x k - x ± , (2.8) 

r •, oo 

for every id , kd . Let P = ip^/cr , 

i d be arbitrary, let 
o 



j v = q * (i v-i ) ’ \ = p v (j v } » 



Substituting i = i ^ » k = i into 



V = l,...,n we obtain 

1 n 

03 > H (P,Q*| i ) + - 7 x. - x. 

n 1 o n l l 



V=1 V 



V-l 



V = 1,2,... 

(2.8) and averaging over 



1 n ^ 2 

But |— 7 x. - x. I = — I x. -x. I ^ — maxflxj} 

n ^ L - l , i J nl l l I n 1 i‘ 

1 v=l v v-l no id 

which tends to zero as n 00 . 

Hence (2.2) holds with £3(0) = 03 and Q* = {q*,q*,...} . 

To prove (2.3) let this time j eJ arbitrary , 



\ = P v (j v-1 ) * j v = q * ( V ’ v = 1 * 2 ** 



and substitute k = i , i = i - into (2.8) . 

V VT J 



Averaging over V = 1, . . .n we obtain 
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w > H n (P,Q*|j o ) 



-[ (l-a)a(i 1 ,j ) + Oa(i 1 , j n ) ] + — £ x. - x. 
l l ,J o 1 ,J 1 n L , 1 1 



V=1 V 



v+1 



the last two terms again tending to zero as n -*• 00 . 

Inequalities (2.4) and (2.5) follow from (2.6a) and (2.7b) 
in analogous manner. 

Finally to establish (2.1) we have from (2.7a,b) 
co = oa (i , q* (i) ) + (l-a)a (p* (q* (i) ) , q* (i) ) + x p *( q *( i )j ~ x ± » ieI > 

from which by setting i = i^ = p*(j ) , J v = q*(i p ,V = 1,2,... 

we have 

co = H (P*,Q*| i ) + - (x. - x. ) , 

n n 1 1 

n o 

and (2.1) follows. The lemma is proved. 

Remark 1 : Investigation of the preceeding proof reveals that a slightly 

stronger statem can be made. In fact we proved that there exists a constant 

oo oo 

C < 00 such that for any P<rP , Qc() 

H n (P,Q*|i o ) < n«j) + £ , (2.2’) 

H n (P*,Q|i o ) > n(a) - ^ , (2.4’) 

for every n = 1,2,..., and similarly for a column - predecessor j . 

This also implies that the convergence in (2.1) is of order 0(l/n) . 

With the aid of the lemma we now prove the main theorem about 
MD games . 

Theorem 1 ; Every MD game G(a) , a€[0,l] has an omega-value and each 

player has a stationary optimal response strategy. Further, for every 
ac[0,l] the omega-value ft (a) is the unique number oa satisfying the 
system (2.6) and P* = {p*} , Q* = {q*} , where p* and q* satisfy 
(2.7a) and (2.7b) respectively, are stationary optimal response strategies. 
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Proof : In view of the previous lemma and the fact that, according to its 



definition, the omega-value must be unique, the theorem will follow as 

long as we prove that the system (2.6) has always a solution. 

To this end notice first that if x, , . . . , x is a solution of 

1 n 

the system of N equations: 

x = min max [Oa(k,j) + (l-0)a(i,j) + x.] 

jeJ iel 1 (2.9) 

1 N 

- - l min max[aa(£,j) + (l-a)a(i,j) + x ] , kel , 

1=1 jeJ iel 



then co , x^ . . . , x N> y^ . . . , y M , where 
1 N 

co = — J min max[aa(£,j) + (l-a)a(i,j) + x.] , 

1=1 jeJ iel 1 

y = max [(l-a)a(i,j) + x ] - co(l-a) , jeJ , 

J iel 

is a solution of the system (2.6) . 

Call temporarily f ^(x^, . . . ,x^) , kel the right-hand side of the k-th 
equation (2.9) . Since 

1 N 

| f k ( x i> • • • » X N ) I -ftl I min max [Oa (k, j ) + (l-a)a(i,j) + x^ 

1=1 jeJ iel 

- min max [Oa(£,j) + (l-a)a(i,j) 4- x.]| 

jeJ iel 1 

1 N 

< — J max | max [Ca(k,j) + (l-a)a(i,j) + x ] 

1=1 jeJ iel 

- max [oa(£,j) + (l-0)a(i,j) + x ]| 
iel 



£ J max max| [aa(k,j) + (l-a)a(i,j)] - [aa(£.,j) + (l-a)a (i , j ) ] | 
SL=1 jeJ iel 

N 

< - J max | a (k, j ) - a(£,j)| < 2a || A || , where 
W 2-=l jeJ 
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|| A || = max max |a(i,j)| , 

iel jeJ 

N 

and since clearly J f, (x , . ..,x ) = 0 , 

k=l 



the vector-valued function f_ = (f^,...,f^) maps the 
hypercube 



(N-l) -dimensional 



N 

C = {(x 1 Xjj) : l x ± = 0 , max|x i | < 2a\\ A j|} 

i=l iel 



into itself. Next for any kel 

I f k ( x 1 > • • • » x N ) - f Jc ( x 1 ' x n') I - I min max 

jeJ iel 

[cra(i,j) + (l-cr)a(k,j) + x.] - min max[aa(i,j) 

1 jeJ iel 



1 N 

+ (l-a)a(k, j) + x^] I + — l min max 

&=1 jeJ iel 



[aa(£,j) + (l-a)a(i,j) + x. 1 ]- min max [cra(£,j) + 

1 jeJ iel 



< max 


x.-x! 


+ i I 


max |x. * — x . 


= 2 max 


x.-x. 1 


iel 


l i 


N 


. T i 1 

lel 1 


iel 


n i 



(l-a)a(i, j) 




Hence the function £_ is also continuous and therefore, by Brouwer Fixed 

Point Theorem there exists (x- , . . . , x XT ) e C such that 

1 N 

X k = f k (x l’“ , ’ X N ) ' k € 1 ’ 

Thus the system (2.9) and consequently the system (2.6) has always a 
solution and the theorem is proved. 

John Danskin defined originally the omega-value of a MD game as 
a limit of ordinary pure values of a sequence of games with perfect in- 
formation ([2], see also [3]). The next theorem shows that Danskin 1 s 
definition and the one used in this paper are equivalent. 
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T heorem 2 : If f2(a) is the omega-value of a MD-game G(a) then for 

every i d 
y o 

1 n 

ft(a) = lim min max ... min max — £ aa(i v _ 1> j^) + (l-a)a (i , j ) , 

n- ii£l " V=1 - ( 2 ^ 

and for every j eJ 



1 a 

fi(o) = lim max min ... max min - [ (l-O)a(i ,j ) + Oa(i ,j ) 

n-*» i £ I j-,£J i el j £J V=1 

11 " 0 ( 2 . 11 ) 

Pr oof : Denote temporarily 



1 n 

W (i ,j-,i 15 ...,j , i ) = — J aa(i n , j ) 4- (l-a)a(i , j ). 
n o ,J l’ 1’ J n n n V-1 ,J V v ,J v 

Let p *eP and q*e 0 be as in (2.7) . By (2.2 ! ) and (2.4 f ) of 
Remark 1 we have for every n = 1,2,... 



W n (i 0 ,q*(i 0 )>ii>q*(ip» q * ^n-l^ ’ ^n^ < + n 

for any sequence of row indices i Q ,i^, ... , and 

W n^ i o’h’ P *^ J l^’ ‘ * *’h’ p *^n^ > " C ^ n ’ 

for any sequence of column indices i^ 9 ^ 9 *** and an y i el 

From (2.12) we obtain successively 

max W n (i o ,q*(i o ),...,q*(i n _ 1 )i n ) < «(o) + ~ 
n 

min max W n (i Q , q* (i Q > , • • • , , j i Q ) < W) + f 

J n n 



( 2 . 12 ) 



(2.13) 



• • . 



min max ... min max W (i ,i* ,i ) < fi(a) H . 

n v o’ J l’ J n n n 

J 1 X 1 J n X n 

Similarly from (2.13) we obtain eventually 

C 

min max ... min max W (i , i j ,i ) > Q(o) H 

n o ,J l’ J n n n 



J i h 



J n X n 



and (2.10) follows. (2.11) is proved in the same fashion. 
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To the end of this section let us investigate some simple properties 
of the omega-value. From the definition or from the previous theorem it 
is immediately obvious that 

£2(0) = min max a(i,j) , 
j£j iel 

£2(1) = max min a(i,j) . 
iel jeJ 

Also, it is easy to see that if the matrix A is reduced by successively 
eliminating dominated rows and columns the omega-value £2(a) is not 
affected. Another obvious property is that if the entries a(i,j) are 
all multiplied by a positive constand a and/or an arbitrary constant 
3 is added to all of them the omega-value changes accordingly while the 
optimal response strategies remain unchanged. Some less obvious properties 
of the omega-value are given in the following theorem. 

Theorem 3 ; The omega-value £2(a) of a MD game is a continuous, non- 
increasing and finite piece-wise linear function of a £ [0,1] . 

Proof : Let a^£[0,l] , c^efO,!] . By Theorem 2 



|^(a 1 ) 


- ft(cr 9 )| = 


|lim[ min max ... min 


max 


z 


n -* 00 J 1 eJ i^el j n eJ 


i £ I 
n 


i n 

1 I o 

n V=1 


i a| Vi'V 


+(l-a i )a(i v , j^) - min 

j l 6J 


max 

i^£l 










• min 

v J 


1 V 

max — T c 

i d " V=1 
n 


r 2 a(i v-r j v )+( 1 “ a 2 )a(i v’ j 


v>ll 
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5 lim 


min max . . 


. min 


1 n 

max — To 


l^V-l’^ 


n-*» 


J 

j^J i^I 


j e J 

J n 


i cl n v=l 
n 


+ (1- 


°l )a( W ■ 


min 


max . . . min 


max 




jj^J 


i,£l j £ J 

1 n 


i £ I 
n 


i n 










1 I 

n L 


a 2 a ^ 1 V-l’^V^ 


+ (1- 


-0 o )a(i , i ) 
2 v J V 


i 


V= 


1 




i n 




< lim 


max max . . . 


max 


max i l | 


(0 r°2 ) 


n^°° 


j l£ J i^I 


v J 


l el V=1 

n 





a (i v _l»j v ) + (°2 



< 2 || A || | a 2 - °2 I 5 where again || A 



= max max | a(i,j) 
iel jeJ 



Hence ft(a) is continuous in Oe[0,l] . 

Next let 

(i i j i W 

be a finite sequence of distinct row and column indices such that 



j v = q*(i v ) , = P*(j v ) , V = , i m+1 = i L . 

where p*eP and q*eQ. satisfy (2.7) . Clearly, such a sequence exists 
for every Oe[0,l] and may be called an optimal cycle. By (2.7) we have 



x i + uo = oa(i ,j v ) + y. 

v J v 

y + (D(l-o) = (l-o)a(i , j ) + x 

J v v+1 



v = 1, . . . ,m 



(2.14a) 

(2.14b) 
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Adding these equations and using the fact that x^ = we obtain 

m+1 



m m 

mw = o J a(i v ,j v ) + (1-0) X a(i vfl ti v ) • 



(2.15) 



Since an optimal cycle exists for every <Je[0,l] and since there is only 
a finite number of possible optimal cycles continuity of fi(CF) implies 
that there must be a finite partition 



0 = cr<a 1 <...<a =1 

o 1 p 



of the interval [cr,l] such that in each interval [cr, ,a,] the omega- 

K.— X K. 

value is a linear function of O , in particular 



m 



m 



ft(a) = — y a 1 ,j + 
7 m t V ,J v' 
V=1 



1-0 



m ^ ' V+1 ,J V' 



l , Vi - a - • (2 - 16) 



Finally, since by 



(2.6b) 

y + to(l-a) > (l-a)a (k, j ) + x k 



for every kcl , jeJ we have by setting j = k = i^ 



y + o)(l -a) > (l-a)a(i ,j ) + x , v = l,...,m , 
J v v 



from which by adding these inequalities to the m equations (2.14a) 
we obtain 

m m 

mo) > a l a(i , j ) + (l-o) l a(i v> j v ) . (2.17) 

v=l v=l 



Comparing (2.17) with (2.15) we see that 
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m m 

and hence by (2.16) ^(0^^) - ^( a ^) • Thus the function fi(a) is 

non-increasing and the theorem is proved. 
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3. Ergodic Pairs and a Method for Solving MDG's 

Given any starting point and any pair of stationary response 

strategies, the (row, column) pair will eventually repeat itself, since there 

are only finitely many such pairs, and a minimal cycle of such pairs will 

then repeat over and over. The payoff to player I will be the average 

payoff over that cycle (2.16). If the cycle turns out not to depend on the 

initial row or column, the pair of stationary response strategies will be 

termed ergodic , and such a pair will be referred to as an ESP. An optimal 

pair of stationary response strategies will be an OSP for whatever values 

of a the pair is optimal. Note that whether or not a pair of strategies 

is an ESP has nothing to do with the payoff matrix or a. 

Since the ft-value does not depend on the starting point, an OSP 

must be an ESP, except when an OSP has multiple cycles each of which has an 

average payoff (formula 2.16) of ft(a). Such OSP T s will turn out to be of 

some importance to our method, but let us ignore them for the moment, and 

attempt to solve for the ft-value (we want the entire function ft(*)» not 

ft(a) for some particular a) using only ESP f s. 

Our method will be to first solve for ft(G) , and then attempt to 

find ft(a) in adjoining intervals until finally we find an interval that 

* 

includes a = 1. When a = 0, ft(0) = min max a(i,j), p ( j ) is a row 

th j6J ifI * 

with the largest payoff in the j — column, and q (i) = j is a min max 

* * * * 
column. (p ,q ) is an ESP, since the only possible cycle with (p ,q ) is 

|(p (j ) , j ) , (p (j ) , j )j. Furthermore, (p ,q ) will be an OSP in some 

maximal interval [ 0 , a ^ ] , where possibly = 0. In order to find a^, 

we must solve equations (2.7) for x, y, and a) as functions of a, after 

which will be the largest value of a for which (2.6) holds. 
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Before solving (2.7), we first note that an arbitrary function of 
a can always be added to both sides of 2.6b and 2.7b. In order to obtain 

linear expressions, it is convenient to let this function be t oa. If we 

» 

then let x_, = x. + cog , 2.6 and 2.7 become 

i i 

f 

x. = min[cr a(i,j) + y.], ifl (3.6a) 

1 j€J 3 

f 

y . + a) = max{(l-a) a(i,j) + x. ] , j(?J (3.6b) 

3 ifl 1 

x ± = o a(i ,q*(i) ) + y ^ (3.7a) 

y^ + a) = (1 -a) a(p (j) ,j) + x 1 (3.7b) 

I I 

In the following, we will drop the on x_^ , and will not refer 

again to (2.6) and (2.7). 

* * — - 
Lemma: Let (p ,q ) be any ESP. Then the solution of (3.7) for x, y, o> 

as functions of a is unique for a), and unique for x, y except 

that one x^ or y can be an arbitrary function of a. If the 

arbitrary function is linear, so are x and y. The quantity co 

is a linear function of o in any case. 



Proof : 



Let the unique cycle be (i^ , j ^ . ,i m> . Let x^ = c(a) , an 

arbitrary function. (3.7a) then defines y. , after which (3.7b) 

3 i 



defines x. , etc. Using this procedure, we can define x(y) 

1 2 

numbers for each row (column) in the cycle. Furthermore, (3.7b) 
with i = j will be satisfied if m is obtained from formula 
(2.16). This defines all x(y) numbers for rows (columns) that 
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appear in the cycle in a manner consistent with (3.7). Let 

i^ be a row that does not appear in the cycle, and let 

* * 



(i 1 > j 1 ,i 2 ,j 2 »- • • be such that = q (i y ) , i v+1 = P (j v >, 

and j is the only column in the cycle (such a will always 

exist, on account of the uniqueness of the cycle). Since y. 

^m 

is defined, we can use (3.7a) to obtain x. , then (3.7b) to 

1 m 

obtain y. , etc., until finally we obtain x. . 

J m-1 1 1 

Furthermore, since any other such sequence that contains i^ will 

also contain there is no danger of obtaining a 

conflicting definition of x^ in the process of defining some 

other x. or y.. Similar remarks hold for columns that do not 
i J 

appear in the cycle. Every x^ and y has therefore been 
uniquely defined in a manner consistent with (3.7). If c(a) is 
linear, the linearity of x and y follows from the linearity 
of (3.7). Since (2.16) is also linear, the lemma is proved. 



We are now ready to describe the process of interval extension, 

* * * 

supposing that ft (a) is already known for 0 <. a ^ o . If the OSP (p ,q ) 
* 



at a 



a is actually an ESP, 



x. 

l 



>y j 



and u) will satisfy (3.6) at 



* 



o = a . Therefore, unless there are ties for the minimum in (3.6a) or the 



maximum in (3.6b) when a = a , the same expressions will satisfy (3.6) 

* ** ** * * * 
for a , where o > a ; that is, (p ,q ) is actually an 

* ** ** 



OSP for o £ a £ a . The quantity a will be the smallest a for 
which there is a tie. For definiteness, suppose there is a tie in (3.6a), 
so that 



x. 

l 



& a 

c a(i,j) + y 



£ ^ & 

= a a(i,q (i)) + y , 
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where j 4 q (i) . We form a new response strategy q (*) by defining 

q (i) = j, and q (k) = q (k) for k 4 i* Then (p ,q ) is an OSP 
** * ** 

at a = a . If it turns out that (p ,q ) is also an ESP, then we can 

repeat the process of solving (3.7) for x^(a) , y^(a) , and <o(a) , 

increasing a until another tie is encountered in (3.6), etc. 

Since we have a starting point (an OSP that is also an ESP at a = 0) , 

there is some hope that the process might map out ft(a) for 0 ^ a ^ 1 in 

intervals, with an OSP/ESP for each interval. Two things are clear: 

1) The process is failsafe in the sense that all answers are correct. 

2) The process may not provide an answer. This could happen either 

because the process gets stuck at a certain a (multiple ties 

* ** 

might cause this) , or because (p ,q ) is at some stage not 
and ESP. 

Computational experience with matrices chosen at random has shown that the 
process will always (?) provide the answer for (3 x 3) matrices, but that 
it will sometimes fail on (9 x 9) matrices and will nearly always fail on 
(18 x 18) matrices. When it fails, it fails because it discovers an OSP 
that is not an ESP. 

This reason for failure is somewhat suprising. If an OSP that is 
not an ESP is to hold over an interval , then (2.16) must be the same 
function of a for two or more disjoint cycles. If the a(i,j) are chosen 
at random, the probability of this is 0 . In fact, the probability is 0 
that there could be an OSP with three or more cycles for any value of a. 
However, the probability is not zero that there can exist particular 
values of a at which an OSP has exactly two disjoint cycles, and it is 
this type of OSP that the above procedure tends to discover. The problem. 
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* ** 

then, is this: Given ESP 1 that is an OSP over some interval [o ,o ] 

and OSP^ at a that is not an ESP, how can we discover an ESP^ that 
is OSP over [o ,a ] where o -> a ? 

From ESP ^ , we can obtain numbers x^,y., and u> that satisfy 
(3.6) at o=a . There is a tie in one of the equations (3.6), which 
we assume for convenience to be in (3.6a): 

** ** 
x = a a(i,j..) + y = a a(i,j ) + y , 

J 1 J 2 

where 

* ** 
j x = q (i) and j 2 = q (i) , 



£ * £ * £ 
ESP X = (p ,q ), and OSPj^ = (p ,q ). 



We 



assume OSP^ has exactly two cycles, one of which must be the ESP^ 



cycle. The rows and columns can be partitioned into and S^, where 

includes all rows and columns in the ESP^ cycle, plus all those rows 



and columns that OSP^ maps into the ESP^ cycle, and 



is defined 



similarly for the other cycle. Define x(6) and y(6) by 



x i (6) = 



x. if ifS 

l 1 



x. - 6 if iC-S 0 

l 2 



y j w > *■ 



Yj if 



- <S if jfS 2 
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Since (3.7) with OSP^ never compares a row in with a column in 

or a row in with a column in S^, x(6),y(6) and OSP^ will solve 

(3.7) regardless of 6. Let 6 be the largest 6 such that x(6),y(6) 
satisfy (3.6). Since we have assumed there is only one tie when 6 = 0, 

6 > 0. For 0 < 6 <6, there are no ties. When 6 = 6, there is at 
least one tie, and we assume there is exactly one. Resolving the tie one 
way, we obtain OSP^. Resolving the tie the other way, we obtain OSP^ 
different from OSP^ and ESP^. If OSP 2 is actually ^^P^, we re-solve 
(3.7) and proceed with interval extension. If OSP 2 not ^SP > then with 

probability one it has the same two cycles as OSP^* since otherwise there 
would be three distinct cycles with the same average payoff in the sense 
(2.16). Let S.^ be all those rows and columns that ^SP^ maps to the 
first (original ESP^) cycle, and similarly for $ 2 ^* Now repeat the 
process of subtracting 6 from every x_^ or y with subscript in 
until still a different tie is revealed, with corresponding OSP^, etc. 
Sooner or later a new ESP will be discovered, and the basic process of 
interval extension can continue. 

The above is not intended to be a proof that the procedure will 
work, but only as an explanation of the process used by a FORTRAN program 
called MATDIF to solve differential games. MATDIF is failsafe, in the 
sense that it deals only with solutions of (3.6) and (3.7), but it has not 
been proved that MATDIF will always provide an answer for all aftOjl]. 
However, MATDIF has not failed to produce an answer for any of the ap- 
proximately 100 test matrices with elements chosen to be uniform random 
numbers between 0 and 100. The program compiles in about 10 secs on the 
NPS IBM360/67, and the run time is approximately 120 (M/50) ^ ^ (N/50) 



19 



seconds (see Table 1). The program is available from Washburn. In the 
future, a proof that MATDIF or a procedure modified to account for 
degeneracies will always provide an answer will be provided. 
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T(M,N) » 120 





N. N 


9 


18 


50 




. 6 seconds 


1.5 seconds 




9 










(20 runs) 


(30 runs) 






1.5 seconds 


4.5 seconds 
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| 


! 

i 

1 


(30 runs) 


( 2 runs) 


. 


i 

i 

i 


10 seconds 




120 seconds : 


50 






i 

1 




(10 runs) 




(3 runs) 
J 



Table 1 Average run times for MATDIF in seconds 
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Example: Figure 1 shows ft(c) for a typical 18 x 18 game. There are 20 

distinct linear segments in this case. MATDIF actually considers 70 
different ESP’s, but many of the (distinct) ESP’s have the same cycle. 

The solution proceeds by interval extension up to a = = .53169. The 

ESP^ that is OSP at and for slightly smaller values is illustrated 

in figure 2; each column has an ”x” corresponding to I’s choice in the 
column, and each row has an ”o” corresponding to II* s choice in the row. 

So x’s move horizontally to o’s, and o’s move vertically to x’s. The only 
cycle is shown solid in the figure. 

The OSP^ following ESP^ is identical to ESP^ except that the 
0 in row 16 is moved from column 11 to column 2. A new cycle forms, shown 
as a dashed line. Both cycles have the same average payoff. The reader 
might amuse himself by delineating S^ and S^‘ OSP 2 f° rmec * by moving 

the 0 in row 6 from column 3 to column 6, and still has the same two cycles. 
OSP^ is formed by moving the x in column 16 from row 16 to row 10, and 
also has the same two cycles. Finally, OSP^ is formed by moving the 0 in 
row 5 from column 17 to column 8, leaving only the dashed cycle and hence 
a new ESP. This ESP is valid for ^ o ^ .5402, and only ESP’s are 
encountered for larger a. The run time for solving this game is 4.12 
seconds . 
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£2_(cr) 




FIGURE 1 

ft(o) for an 18 x 18 game 






/ 2 3 y s' <0 7 t f to »/ /Z 13 /? IS /6> 17 *9 




FIGURE 2 



Illustration of cycles at a = .53169 
with same game as in FIGURE 1. 
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